The present invention relates to volume imaging, in particular to a novel method and apparatus for registering volume images.
The term "image", as used herein, refers to a volume image, unless otherwise indicated. A volume image is the three-dimensional (3D) analogue to a two-dimensional (2D) image. A 2D image can be represented for digital image processing purposes as a collection of pixels, where each pixel has a location (usually specified by x,y coordinates) and a color value. An image is formed on a display (or printer) when the pixels (or print dots) are colored according to their color value and placed at their location.
A 2D image is a representation of a flat view, as might be found on a computer monitor, newspaper, X-ray, or the like. Although the 2D image might be of a 3D object, the image itself is 2D, because of the way it is captured. For example, an X-ray is a 2D image because it contains no depth information, even though the X-ray might be of a patient's internal organs (a 3D object). If a 3D image of a 3D object is desired, a volume image should be captured instead of a 2D image.
The term "pixel", as used herein, will at times refer to an actual display element, such as a single picture element of a computer display, but at other times will refer to an identified location on a printed page where a dot of color could be placed, or a location on a image which exists only in the memory of a computer as an array of pixel values. Typically, where the image is stored in a memory of a computer, the pixel locations are not stored, but are implied from the pixel's position in the dataset, file, or memory block in which the "image" is stored.
The term "pixel color value", or "color" for short, refers to the value of the pixel, which can be a color or a shade of gray, depending on the palette. Common palettes include 24-bit color, 8-bit monochrome (shades of gray) and one bit color. One-bit color is often referred to as "black and white", but it should be apparent that any other assignment of two colors to the two possible color values would be equivalent. A one-bit color image is often referred to as a binary image, since all of the pixels take on one of two color values.
The capturing of volume images is well known in the medical field. MR (magnetic resonance) techniques and tomography are but two examples of volume image capture methods. A volume image is typically stored in a computer memory as an array of "voxels" where each voxel has a location in 3D space (e.g., x, y and z coordinates) and a voxel color value. Where the volume image is a monochrome image, the voxel color value is the "intensity" of the voxel. Thus, a voxel is the basic element of a volume (3D) image in the same way that a pixel is the basic element of a 2D image.
While a volume image might be displayed on a 2D display, it is nonetheless stored as a volume image in the computer memory. One advantage of a volume image over a 2D image is that the volume image can be displayed in three dimensions on a 3D display, such as might be used by a doctor or radiologist to noninvasively examine internal organs of a patient. Even if a 3D display device is not used, volume images have another advantage over 2D images, in that the position of a 2D "slice" through the volume image displayed on a 2D display device can be changed. Often a diagnostician can get a good idea of what the 3D image of the internal organs looks like by just viewing a 2D image slice and manipulating the position of the slicing plane.
"Registration" refers to the process of aligning two or more related images. These images can represent the same data type or different data types (such as proton density MR, T2-weighted MR, PET, etc.). This data is typically taken of some patient, but possibly from differing views (distance and angle). Registration can also be used to align two or more subimages which contain the same type of data but are multiple images taken with partial overlaps. This latter application is akin to the process of taking multiple vacation photographs of a landscape and taping them together to form a large panoramic collage. For such a panorama effect to work, the photographs must be aligned ("registered") so that the scene is continuous from one photograph to the next. To register the photographs, there must be some image overlap from photograph to photograph.
Automatic registration, such as might be performed by a digital computer is known. The input to such an automatic registration process is two images and the output of the process is a transformation matrix. The transformation matrix is a set of parameters which indicates how one of the images should be transformed relative to some fixed coordinate system to bring the two images into alignment. Of course, the output of the process could just be the combination of the two images, but it is often more compact and useful to store just the transformation matrix. With more than two images, registration can be performed serially, with the third and subsequent images registered to the first in order to allow accurate time or other variation analysis.
With 2D images, a transformation matrix need only specify a planar transformation. With volume images, a transformation matrix is more complicated, as it needs to specify a transformation in a 3D space. A planar transformation can include a rotation within the plane, translation in one or two directions, and a scaling in one or more directions, while a 3D transformation can comprise their analogues in a coordinate space.
Registering volume images is increasingly important as medical tomography and MR become more and more prevalent diagnostic tools. Researchers are discovering that some useful information only exists through the registration and combination of volumes containing object information. For example, a patient might undergo one scan with a particular paramagnetic contrast material administered to the patient and a certain MR sequence, then undergo a second scan with different conditions. The results of these two scans, when overlaid and registered, might provide insights to the diagnostician where none could be obtained by viewing the results separately. For examples of experiments using multiple MR sequences, see (1) V. Dousett, et al., "Experimental Allergic Encephalomyelitis and Multiple Sclerosis: Lesion Characterization with Magnetic Transfer Imaging", Radiology 182:483-491 (1992). "Sequence" refers to the particular pulse sequence used for an MR scan.
One inherent problem with automatic registration of images created using different imaging methods is that the voxel intensities used to represent a particular tissue using one method are often different and unrelated to the voxel intensities used by another method. The voxel intensities might be different due to different imaging modality, MR sequence parameters or changes in the patient's physiology from one scan to the next. This can cause problems for registration algorithms which expect a specific relationship between intervolume voxel intensities.
There are many other techniques that attempt to perform image registration by assuming specific relationships among voxel intensities, such as R. Woods, et al., "MRI-PET Registration with Automated Algorithm", Journal Computer Assisted Tomography 17(4):536-546 (July/August 1993; hereinafter "Woods I"), describes one solution to the problem of volume registration. There, the goal was to register MRI and PET volumes of the brain. The problem with MRI and PET volumes is that different types of tissue have different intensities, and the intensities bear no simple relationship. The solution provided by Woods I is to manually edit the MR images to remove nonbrain structures. This manual editing is time consuming and allows the registration to be too user dependent. The remaining image is partitioned into components based on the values of the MRI voxels in a way which maximizes the uniformity of the PET voxels in each component, but the assumption there is that all MRI voxels with a particular value represent the same type of tissue, so the corresponding PET voxels should have similar values to each other. See also, R. Woods, et al., "Rapid Automated Algorithm for Aligning and Reslicing PET Images", Journal Computer Assisted Tomography 16(4):620-633 (July/August 1992) (hereinafter "Woods II") and M. Herbin, et al., "Automated Registration of Dissimilar Images: Application to Medical Imagery", Computer Vision, Graphics, and Image Processing 47:77-88 (1989).
One way to avoid the problem of registering multiple images or multiple subimages with differing intensity levels is to estimate the orientation of the structure in 3D space instead of attempting to match voxel intensity, as shown by N. Alpert, et al., "The Principal Axes Transformation--A Method for Image Registration", Journal of Nuclear Medicine 31:1717-1722 (October, 1990). Unfortunately, changes in patient placement create regions of uncommon data which adversely affect these types of algorithms.
As shown by C. A. Pelizzari, et al., "Accurate Three-Dimensional Registration of CT, PET and/or MR Images of the Brain", Journal of Computer Assisted Tomography 13(1):20-26 (January/February 1989); H. Jiang, et al., "Image Registration of Multi-Modality 3-D Medical Images by Chamfer Matching", SPIE Vol. 1660 Biomedical Image Processing and Three-Dimensional Microscopy 356-366 (1992) and P. J. Besi, et al., "A Method for Registration of 3-D Shapes", IEEE Transactions on Pattern Analysis and Machine Intelligence 14(2):239-256 (February 1992), contour methods will avoid differences in tissue-based voxel intensity relations, but such methods ignore a large amount of internal object information which is useful for calculation of an accurate transformation matrix. This information is particularly important when one considers the limitations of contours due to occlusion, partial volume effects and image noise.
Yet another approach is to use voxel intensity as the substrate for more complicated comparison metrics to guide coregistration. Typically, these methods require substantial image processing, considerable computing time and are theoretically complex, making it difficult to understand why coregistration might fail in any particular instance.
Therefore, what is needed is improved registration of multiple volume images or multiple volume subimages which works even when the images to be registered have no direct relationship between intensity levels, varying patient placement (in the case of medical images), occlusion and noise.